Skip to content

Add medium-range lagged ensemble with multiprocessing - #52

Merged
mxkpp merged 11 commits into
developmentfrom
maxkipp-laggedens-n2
Apr 24, 2026
Merged

Add medium-range lagged ensemble with multiprocessing#52
mxkpp merged 11 commits into
developmentfrom
maxkipp-laggedens-n2

Conversation

@mxkpp

@mxkpp mxkpp commented Apr 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

This adds initial support to RTE for running a medium-range lagged ensemble forecast, including multiprocessing. Currently the lagged ensemble workflow's optional args open_loop_state and closed_loop_state are not used by RTE and must be empty parts of the --lagged-ensemble CLI arg. Support for these can be added later.

The lagged ensemble is treated as a medium_range forcing configuration type, and each member is a separate call to run_forecast.py or run_default.py

This should be reviewed in conjunction with related PRs on other repositories:

NGWPC/nwm-msw-mgr#54 (already merged)

NGWPC/nwm-fcst-mgr#18 (closed)

NGWPC/ngen-forcing#138 (already merged)

nwm-fcst-mgr branch: https://github.com/NGWPC/nwm-fcst-mgr/tree/jwade_NGWPC-10211_wcoss

nwm-msw-mgr branch: https://github.com/NGWPC/nwm-msw-mgr/tree/jwade_NGWPC-10183_update_fcst

Update: For run_forecast.py calls to lagged ensemble, the development branches of nwm-msw-mgr and nwm-fcst-mgr may be used. The above non-development branches are only needed for run_default.py calls to lagged ensemble at this time.

This was referenced Apr 13, 2026
Merged
Closed
Merged
@mxkpp
mxkpp marked this pull request as ready for review April 13, 2026 15:20
@mxkpp
mxkpp requested review from jswade-rtx and kyle-larkin April 13, 2026 15:20
@mxkpp
mxkpp marked this pull request as draft April 16, 2026 20:18
@mxkpp
mxkpp marked this pull request as ready for review April 17, 2026 19:14
@mxkpp

mxkpp commented Apr 17, 2026

Copy link
Copy Markdown
Contributor Author

@kyle-larkin, @jswade-rtx this is now ready for review. It currently requires these branches

nwm-fcst-mgr branch: https://github.com/NGWPC/nwm-fcst-mgr/tree/jwade_NGWPC-10211_wcoss

nwm-msw-mgr branch for this new dict:

https://github.com/NGWPC/nwm-msw-mgr/tree/jwade_NGWPC-10183_update_fcst

@mxkpp

mxkpp commented Apr 17, 2026

Copy link
Copy Markdown
Contributor Author

Note for reviewers and testers: you will also need to pull the latest ngen-forcing development branch, to have the updated forcing configuration files available to be mounted at runtime.

@mxkpp

mxkpp commented Apr 21, 2026

Copy link
Copy Markdown
Contributor Author

@cosumi-rtx if looking to test the lagged ensemble members through run_forecast.py aka run_fcst.sh, i.e. for a realtime forecast of a gage that already has its dependencies met (already has a calibration result) then I believe the "forecast" workflow will currently work with development branches of nwm-msw-mgr and nwm-fcst-mgr. I just tested this and the realizations did start. I did not let them run to completion.

Though if wanting to run the "default" workflow (run_default.py aka run_default.sh`) methods of lagged ensemble members, where there is no dependency on a preexisting calibration result, then non-development branches mentioned in earlier comments of this PR are needed currently.

@mxkpp

mxkpp commented Apr 24, 2026

Copy link
Copy Markdown
Contributor Author

Merging after confirming with team

@mxkpp
mxkpp merged commit d9fbe27 into development Apr 24, 2026
7 of 8 checks passed
@mxkpp
mxkpp deleted the maxkipp-laggedens-n2 branch April 24, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kyle-larkin kyle-larkin Awaiting requested review from kyle-larkin
@jswade-rtx jswade-rtx Awaiting requested review from jswade-rtx

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mxkpp